1. Statement of the Technical Field
The inventive arrangements concern a new correlation technique that can be used to determine the degree of similarity between data sets. More particularly, the invention concerns correlation techniques which are relatively insensitive to rotational variations occurring in data sets such as image data and perform better than traditional correlation methods on data from disparate sensors.
2. Description of the Related Art
Image data for a particular scene is often obtained by one or more sensors at different times or from different perspectives. Consequently, the image data associated with each image will generally be defined in a different coordinate system due to the different perspective of the sensor when each such image is obtained. In other words, the point of view of the sensor may be different in each case. It is often desirable to combine two or more such images to create a composite image. However, the problem of combining the various images to form a single composite image can be difficult.
Computers can be programmed with various mathematical algorithms to combine image data of the same scene but obtained from different sensor perspectives. The term “image registration” refers to the process of transforming the different sets of data into a common coordinate system. Image registration is necessary in order to be able to compare or integrate the image data obtained from the same sensor in different positions or from different sensors at different times.
Various correlation methods are known in the art for purposes of performing image registration. One such method is known as the phase correlation method. The phase correlation method uses properties of the frequency domain to determine shifts between two images. Applying the Phase correlation method to a first and second image of a common scene will result in a correlation surface that ideally contains a single peak. Advantages of the phase correlation method are that a sharp peak is produced when the images are aligned and its robustness under noise and occlusions.
A second correlation technique for image registration is based on image similarity in the spatial domain (although a frequency domain implementation is well-known). It is referred to as normalized cross-correlation. A normalization step makes it invariant to illumination differences. An advantage of this method is the relatively slowly varying correlation surface. A disadvantage is that the peak is difficult to detect in the presence of noise. Other similarity metrics include mean absolute difference (MAD) and sum of squared differences, (SDD).
In the case of image alignment or registration, corresponding subregions within the overlapping area between two data sets are identified. A subregion is some area that is less than or equal to the overlapping region between the two data sets. For example, the overlapping area contained in each one of the two data sets can be divided into a plurality of subregions. Data in each of the corresponding subregions undergoes a correlation process. Conceptually this can be described as follows though the actual implementation may vary including a frequency domain implementation as is well known in this field. A small patch is formed about the center of the target image subregion. A larger patch is formed about the center of the reference image subregion. A patch is a two-dimension matrix of image pixels. To find the correlation between the two patches, the small patch is positioned over the large patch, the corresponding pixel values are multiplied, summed, and normalized. This value is called the correlation score. This is repeated at every location within the large patch in a sliding window manner. The correlation scores are saved in a two dimension grid called a correlation surface. The peak of this surface corresponds to the region within the reference patch that the target patch is most similar. The center location of the target patch along with it's best fit inside the reference patch are saved in the coordinate system of the original images. This process is performed for each subregion of interest. For each subregion, the location of the maximum correspondence is saved creating corresponding point sets. Finally, an optimization algorithm is used to minimize the distance between the corresponding point sets by changing parameters associated with a transformational model, for example an affine or polynomial warping.
Although the phase correlation method and the normalized cross-correlation method can be effective, both are sensitive to variations in the angle of the sensor relative to the scene for which the image data has been collected as well as sensor phenomenology differences. Both techniques demonstrate relatively poor performance in those cases where variations in sensor orientation respectively associated with two image pairs to be registered are greater than about three degrees.